Learning data creation support apparatus, learning data creation support method, and learning data creation support program

ABSTRACT

Image-before-biopsy acquisition means acquires, as an image before biopsy, an image of a biopsy target part of a patient captured at a date and time within a predetermined range before an examination date and time of the biopsy. In a case in which a biopsy result indicated by biopsy data indicates a lesion, registration means registers a lesion candidate obtained by performing an image analysis process for the image before biopsy as correct answer data.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 119(a) to PatentApplication No. 2018-037977 filed in Japan on Mar. 2, 2018, all of whichare hereby expressly incorporated by reference into the presentapplication.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a learning data creation supportapparatus, a learning data creation support method, and a learning datacreation support program.

2. Description of the Related Art

In recent years, machine learning has been used in order to learn thecharacteristics of data and to recognize and classify, for example,images. In recent years, various learning methods have been developedand the processing time has been reduced with the improvement of theprocessing capability of computers. Therefore, it is possible to performdeep learning in which a system learns the characteristics of, forexample, image data at a deeper level. The performance of deep learningmakes it possible to recognize the characteristics of, for example,images with very high accuracy. The discrimination performance isexpected to be improved.

In a medical field, artificial intelligence (AI) which performs learningusing a deep learning method to recognize the characteristics of animage with high accuracy is preferable. In the deep learning, it isnecessary to perform learning with a large amount of high-quality datasuitable for the purpose. Therefore, it is important to efficientlyprepare learning data. In each medical institution, image data for manycases is accumulated with the spread of a picture archiving andcommunication system (PACS). Therefore, a technique has been examinedwhich performs learning using image data for various cases accumulatedin each medical institution.

In addition, in the medical field, whether an image is a correct answeror an incorrect answer is learned using an image of a shadow, such as atumor appearing on an image, as learning data to discriminate the shadowappearing on the image. For example, JP2008-523876A discloses a methodin which the actual image of a patient having different types of lesionsis interpreted by a plurality of experts, such as radiologists, andlearning data is created on the basis of a plurality of types of imagesand the interpretation result of the images by a plurality of reliableand skilled radiologists to improve the reliability of the learningdata.

SUMMARY OF THE INVENTION

However, in this method, it takes a lot of time and effort for a personto find the image to be registered as learning data while checking aplurality of images and it is difficult to register a large amount oflearning data.

In contrast, in a normal medical examination, in a case in which adisease is suspected, the radiologist interprets an image, such as a CTimage or an MRI image, of the part corresponding to the suspecteddisease. In a case in which there is a suspicious finding, thepathologist performs a thorough examination on the part and makes adefinite diagnosis. In a case in which there is a thorough examinationresult (for example, a pathological result) of a suspicious part foundin the image, an abnormal shadow region obtained by an image analysisprocess can be confirmed as correct answer data.

In order to solve the above-mentioned problems, an object of theinvention is to provide a learning data creation support apparatus, alearning data creation support method, and a non-transitory computerreadable recording medium storing a learning data creation supportprogram that can create learning data required for learning withoutperforming a complicated operation.

According to an aspect of the invention, there is provided a learningdata creation support apparatus comprising: biopsy data acquisitionmeans for acquiring biopsy data indicating a result of a biopsyperformed for a patient; image-before-biopsy acquisition means foracquiring, as an image before biopsy, an image of a biopsy target partof the patient captured at a date and time within a predetermined rangebefore an examination date and time of the biopsy; and registrationmeans for, in a case in which the biopsy result indicated by the biopsydata indicates a lesion, registering, as correct answer data, an imageof a lesion candidate region obtained by performing an image analysisprocess for the image before biopsy or the image before biopsy.

According to another aspect of the invention, there is provided alearning data creation support method performed in a learning datacreation support apparatus. The learning data creation support methodcomprises: a biopsy data acquisition step of acquiring biopsy dataindicating a result of a biopsy performed for a patient; animage-before-biopsy acquisition step of acquiring, as an image beforebiopsy, an image of a biopsy target part of the patient captured at adate and time within a predetermined range before an examination dateand time of the biopsy; and a registration step of, in a case in whichthe biopsy result indicated by the biopsy data indicates a lesion,registering, as correct answer data, an image of a lesion candidateregion obtained by performing an image analysis process for the imagebefore biopsy or the image before biopsy.

According to still another aspect of the invention, there is provided anon-transitory computer recording medium storing a learning datacreation support program that causes a computer to perform: a biopsydata acquisition step of acquiring biopsy data indicating a result of abiopsy performed for a patient; an image-before-biopsy acquisition stepof acquiring, as an image before biopsy, an image of a biopsy targetpart of the patient captured at a date and time within a predeterminedrange before an examination date and time of the biopsy; and aregistration step of, in a case in which the biopsy result indicated bythe biopsy data indicates a lesion, registering, as correct answer data,an image of a lesion candidate region obtained by performing an imageanalysis process for the image before biopsy or the image before biopsy.

The “lesion candidate region” means a region that is likely to be alesion on the image and indicates a region that is more likely to be alesion extracted by the image analysis process of the computer thanother regions in the image.

The “correct answer data” means data in which a correct answer label isattached to the lesion candidate region. In a case in which the lesioncandidate region is determined to be an image of a lesion, the lesioncandidate region is correct answer data. Alternatively, the “correctanswer data” means data in which a correct answer label is attached toan image before biopsy. In a case in which the image before biopsy isdetermined to be an image indicating a lesion, the image before biopsyis correct answer data.

Preferably, in a case in which the image of the lesion candidate regionis registered, the registration means specifies a correspondenceposition corresponding to a position where the biopsy has been performedin the image before biopsy from two partial images of the target partcaptured in different directions. Preferably, in a case in which thebiopsy result indicates a lesion, the registration means registers, asthe correct answer data, an image of a lesion candidate region at aposition matched with the correspondence position among positions of aplurality of lesion candidate regions obtained from the image beforebiopsy.

For the “matching between the position of the lesion candidate regionand the correspondence position”, in a case in which the positions arewithin a predetermined reference range, the positions may be determinedto be matched with each other. In addition, the positions may be withina range that is appropriately determined according to the performance ofthe computer and the resolution of the image.

Preferably, the learning data creation support apparatus may furthercomprise learning means for, in a case in which the image of the lesioncandidate region is registered, detecting a lesion candidate region,using a discriminator that has learned the correct answer data of theimage of the lesion candidate region, and directing the discriminator torelearn the correct answer data of the registered image of the lesioncandidate region.

The learning data creation support apparatus may further comprise:learning means for, in a case in which the image before biopsy isregistered, detecting whether an image indicating a lesion is present,using a discriminator that has learned the correct answer data of theimage before biopsy, and directing the discriminator to relearn thecorrect answer data of the registered image before biopsy.

The image of the biopsy target part may be a mammographic image.

According to yet another aspect of the invention, there is provided alearning data creation support apparatus comprising: a memory thatstores commands for causing a computer to perform processes; and aprocessor that is configured to execute the stored commands. Theprocessor acquires biopsy data indicating a result of a biopsy performedfor a patient, acquires, as an image before biopsy, an image of a biopsytarget part of the patient captured at a date and time within apredetermined range before an examination date and time of the biopsy,and registers, as correct answer data, a lesion candidate regionobtained by performing an image analysis process for the image beforebiopsy in a case in which the biopsy result indicated by the biopsy dataindicates a lesion.

According to the invention, it is possible to create learning datarequired for learning, without performing a complicated operation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of the schematicconfiguration of a medical information system.

FIG. 2 is a diagram schematically illustrating an example of theconfiguration of a mammography apparatus according to this embodiment.

FIG. 3 is a plan view schematically illustrating a compression plate asviewed from the upper side.

FIG. 4 is a diagram illustrating an example of an aspect in which aradiation emitting unit is inclined to the left and right sides in adirection along an arm.

FIG. 5 is a diagram illustrating an example of the relationship betweenthe position of the radiation emitting unit and two scout images.

FIG. 6 is a diagram illustrating an example of the schematicconfiguration of a learning data creation support apparatus.

FIG. 7 is a flowchart illustrating an example of the flow of an imagingprocess of the mammography apparatus.

FIG. 8 is a flowchart illustrating an example of the flow of a learningdata registration process.

FIG. 9 is a flowchart illustrating an example of the flow of a processof causing a discriminator to learn.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, a first embodiment of the invention will be described withreference to the drawings. FIG. 1 illustrates the schematicconfiguration of a medical information system 1 provided with a learningdata creation support apparatus according to the first embodiment of theinvention.

The medical information system 1 is used to capture an image of anexamination target part of a subject and to store the image on the basisof an examination order from a doctor in a diagnosis and treatmentdepartment, is used by a radiologist in a radiology department tointerpret the captured image and to make an interpretation report, isused for a pathology examination, such as a biopsy, and is used to makea pathology report. As illustrated in FIG. 1, the medical informationsystem 1 is configured by connecting a modality 2, a radiologistworkstation 3, a diagnosis and treatment department workstation 4, animage database 5, an interpretation report database 6, an imageprocessing server 7, a pathology examination workstation 8, and apathology report database 9 so as to communicate with each other througha network 10. An application program that causes each apparatus tofunction as a component of the medical information system 1 is installedin each apparatus. In addition, the application program may be installedfrom a recording medium, such a CD-ROM, or may be downloaded from astorage device of a server connected through a network, such as theInternet, and then installed.

The modality 2 includes an apparatus that captures an image of anexamination target part of the subject, generates an examination imageindicating the part, adds accessory information defined by a digitalimaging and communication in medicine (DICOM) standard to the image, andoutputs the image. Examples of the modality include a mammographyapparatus, a computed tomography (CT) apparatus, a magnetic resonanceimaging (MM) apparatus, a positron emission tomography (PET) apparatus,an ultrasound apparatus, and a computed radiography (CR) apparatus usinga flat panel detector (FPD).

The radiologist workstation 3 (hereinafter, the radiologist workstationis referred to as a radiologist WS 3) is a computer that is used by aradiologist in the radiology department to interpret an image and tomake an interpretation report. The radiologist WS 3 comprises knownhardware configurations, such as a central processing unit (CPU), a mainstorage device, an auxiliary storage device, an input/output interface,a communication interface, an input device, a display device, and a databus. For example, a known operation system is installed in theradiologist WS 3. The radiologist WS 3 includes one high-definitiondisplay or a plurality of high-definition displays as the displaydevice. In the radiologist WS 3, processes, such as the display of animage received from the image database 5, the display of the detectionresult of a portion that appears to be a lesion in an image by an imageprocessing server, and the making of an interpretation report, areperformed by executing software programs for each process. In addition,in the radiologist WS 3, the input device including, for example, akeyboard and a mouse, is used to input an interpretation report on aninterpretation target. In a case in which the input of theinterpretation report is completed, the radiologist WS 3 requests theregistration of the interpretation report to the interpretation reportdatabase 6 through the network 10.

For example, the diagnosis and treatment department workstation 4(hereinafter, the diagnosis and treatment department workstation isreferred to as a diagnosis and treatment department WS 4) is a computerthat is used by a doctor in the diagnosis and treatment department toobserve an image in detail, to read an interpretation report, to read anelectronic chart, and to input an electronic chart. The diagnosis andtreatment department WS 4 comprises known hardware configurations, suchas a CPU, a main storage device, an auxiliary storage device, aninput/output interface, a communication interface, an input device, adisplay device, and a data bus. For example, a known operation system isinstalled in the diagnosis and treatment department WS 4 and thediagnosis and treatment department WS 4 includes one display or aplurality of displays as the display device. In the diagnosis andtreatment department WS 4, processes, such as the display of an imagereceived from the image database 5, the automatic detection or highlightdisplay of a portion that appears to be a lesion in the image, and thedisplay of an interpretation report received from the interpretationreport database 6, are performed by executing software programs for eachprocess.

A software program that provides the functions of a database managementsystem (DBMS) to a general-purpose computer is incorporated into theimage database 5 and the image database 5 comprises a storage. Thestorage may be a hard disk drive, a network attached storage (NAS)connected to the network 10, or a disk device connected to a storagearea network (SAN). In addition, image data captured by the modality 2is transmitted to the image database 5 through the network 10 accordingto a storage format and a communication standard based on the DICOMstandard and is then stored in the image database 5. The examinationimages of a plurality of patients captured by the modality 2 andaccessory information are registered in the image database 5. Forexample, an image identification (ID) for identifying an individualimage, a patient ID for identifying a subject, the examination date andtime when the examination image was generated, the type of modality usedin the examination for acquiring the examination image, patientinformation including the name, age, and sex of the patient, and anexamination part (imaging part) are recorded on the accessoryinformation.

A software program that provides the functions of a database managementsystem (DBMS) to a general-purpose computer is incorporated into theinterpretation report database 6 and the interpretation report database6 comprises a high-capacity storage. The storage may be a high-capacityhard disk drive, a NAS connected to the network 10, or a disk deviceconnected to a SAN. For example, an interpretation report on whichinformation including an image ID for identifying an interpretationtarget image, a radiologist ID for identifying the radiologist who hasinterpreted images, a lesion name, a lesion region, the positionalinformation of a lesion, and the doctor's opinion has been recorded isregistered in the interpretation report database 6.

The image processing server 7 is a computer that is used to performvarious image analysis processes for the image captured by the modality2 and comprises known hardware configurations, such as a CPU, a mainstorage device, an auxiliary storage device, an input/output interface,a communication interface, an input device, a display device, and a databus. For example, a known operation system is installed in the imageprocessing server 7. In addition, for example, the image processingserver 7 performs the image analysis process to detect a region thatappears to be a lesion in the image in response to a request from theradiologist WS 3 or the diagnosis and treatment department WS 4.

The pathology examination workstation 8 (Hereinafter, the pathologyexamination workstation is referred to as a pathology examination WS 8)is a computer that is used by a pathologist to read a pathology reportbased on a biopsy result and to read and input a pathology report. Thepathology examination WS 8 comprises known hardware configurations, suchas a CPU, a main storage device, an auxiliary storage device, aninput/output interface, a communication interface, an input device, adisplay device, and a data bus. For example, a known operation system isinstalled in the pathology examination WS 8. The pathology examinationWS 8 includes a display as the display device. The pathology examinationWS 8 executes a software program for each process, such as the making ofa pathology report, to perform each process. In a case in which theinput of the pathology report is completed, the pathology examination WS8 requests the registration of the pathology report to the pathologyreport database 9 through the network 10.

A software program that provides the functions of a database managementsystem (DBMS) to a general-purpose computer is incorporated into thepathology report database 9 and the pathology report database 9comprises a high-capacity storage. The storage may be a high-capacityhard disk drive, a NAS connected to the network 10, or a disk deviceconnected to a SAN. For example, a pathology report on which informationincluding a patient ID, a pathologist ID for identifying a pathologist,a lesion name, the positional information of a lesion, and the doctor'sopinion has been recorded is registered in the pathology report database9.

The network 10 is a local area network that connects various apparatusesin the hospital. In a case in which the radiologist WS 3 or thepathology examination WS 8 is installed in another hospital or a medicaloffice, the network 10 may be configured by connecting the local areanetworks of each hospital with the Internet or a leased line. In anycase, an optical network is given as an example of the network 10.

Hereinafter, in this embodiment, a specific example in which a biopsyposition is specified from a mammographic image of the breast of apatient and then a biopsy is performed will be described. In thisembodiment, the modality 2 is described as a mammography apparatus 2.Here, the mammography apparatus 2 according to this embodiment will bedescribed in detail with reference to FIGS. 2 to 5.

For example, as illustrated in FIG. 2, in the mammography apparatus 2, aradiation accommodation unit 16 that accommodates a radiation emittingunit 17 and an imaging table 14 are connected to an arm 13 so as to faceeach other. An image recording medium, such as a flat panel detector 15,is set in the imaging table 14 while being accommodated in a recordingmedium holding unit, such as a cassette. The arm 13 is attached to abase 11 by a C-axis 12. In addition, the C-axis 12 having a rotationcenter at the center position of the flat panel detector 15 is attachedsuch that the rotation center of the arm 13 is the center of the flatpanel detector 15 in the X-axis direction (see FIG. 3) and the arm 13 isprovided in the base 11.

The base 11 is provided with an operation unit 28 that is used by anoperator to adjust the height of the imaging table 14 (that is, theheight of the arm 13) and the inclination of the imaging table 14 (thatis, the inclination of the arm 13) and an arm control unit 31 that movesthe arm 13 in the vertical direction and rotates the arm 13 in responseto an input from the operation unit 28.

The arm control unit 31 rotates the C-axis 12 attached to the base 11 toadjust the inclination of the arm 13 and moves the arm 13 in thevertical direction to adjust the height of the imaging table 14.

A compression plate 18 that is provided above the imaging table 14 andcompresses the breast, a support unit 20 that supports the compressionplate 18, and a moving mechanism 19 that moves the support unit 20 alongthe arm 13 in the vertical direction are provided in a central portionof the arm 13. The position and compression force of the compressionplate 18 are controlled by a compression plate controller 34.

FIG. 3 is a top view illustrating the compression plate 18. Asillustrated in FIG. 3, the compression plate 18 comprises an openingportion 25 having a square shape with a size of about 10 cm×10 cm suchthat a biopsy is performed with the breast fixed by the imaging table 14and the compression plate 18.

For example, a biopsy unit 26 illustrated in FIG. 2 comprises a biopsyneedle 21 that is inserted into the breast, a biopsy needle unit 22, anda moving mechanism 24 that moves the biopsy needle unit 22 in the X, Yand Z directions. The position of the tip of the biopsy needle 21 of thebiopsy needle unit 22 is controlled by a needle position controller 35provided in the moving mechanism 24. In FIG. 3, the horizontal directionis the X direction, the vertical direction is the Y direction, and adirection perpendicular to the XY plane is the Z direction.

The mammography apparatus 2 acquires scout images captured in twodirections so as to include a target region of the breast to be biopsiedbefore needling. The scout images are images that are viewed fromdifferent view points in order to check a pathology examinationposition. For example, as illustrated in FIG. 4, two partial imagescaptured in the directions (which are inclined by, for example, 15° ineach of the +θ and −θ directions illustrated in FIG. 4) in which theradiation emitting unit 17 is inclined from a direction along the arm 13to the left and right sides are referred to as the scout images. Inaddition, the scout images are stored in the image database 5 throughthe network 10.

FIG. 5 is a diagram illustrating an example of the relationship betweenthe position of the radiation emitting unit 17 and two scout images. Ina scout image Img1 captured with the radiation emitting unit 17 locatedat a position P1, the position of a target T leans to the left. In ascout image Img2 captured with the radiation emitting unit 17 located ata position P2, the position of the target T leans to the right. Adistance z from the bottom (the side to which the breast is pressed) ofthe compression plate 18 to the target T and the position of the targetT in the XY plane can be calculated from the positional deviation of thetarget T between two scout images Img1 and Img2 and thethree-dimensional positional information of the target T can beobtained.

In a case in which the positional information of the target T isreceived, the needle position controller 35 of the biopsy unit 26 movesthe tip of the biopsy needle 21 to the position of the target T andinserts the biopsy needle 21 into the breast.

Next, the learning data creation support apparatus according to thisembodiment will be described with reference to FIG. 6. In thisembodiment, a learning data creation support program is installed in theimage processing server 7 such that the image processing server 7functions as the learning data creation support apparatus according tothe invention. The learning data creation support program may be storedin a storage device of a computer connected to the network or a networkstorage so as to be accessed from the outside, may be downloaded to theimage processing server 7, and then may be installed in the imageprocessing server 7. Alternatively, the learning data creation supportprogram may be recorded on a recording medium, such as a digitalversatile disc (DVD) or a compact disc read only memory (CD-ROM), may bedistributed, and may be installed in a computer from the recordingmedium.

In a case in which the image processing server 7 starts up, the learningdata creation support program is stored in a main storage device (notillustrated) and a CPU 70 performs the processes according to theprogram stored in the main storage device. Hereinafter, the imageprocessing server 7 will be described as a learning data creationsupport apparatus 7.

Next, the functions of the learning data creation support apparatus 7will be described. For example, as illustrated in FIG. 6, the learningdata creation support apparatus 7 functions as biopsy data acquisitionmeans 71, image-before-biopsy acquisition means 72, image analysis means73 (hereinafter, referred to as an image analysis process 73), scoutimage acquisition means 74, correspondence position determination means75, registration means 76, a learning data storage unit 77, and learningmeans 78.

The biopsy data acquisition means 71 acquires biopsy data indicating theexamination result of the tissues collected by a needling operation ofthe mammography apparatus 2 from the pathology report database 9.Information including a patient ID, an examination date and time, atarget part (for example, the breast), and the position of a target(information of, for example, an inner upper part, an inner lower part,an outer upper part, and an outer lower part of the breast in additionto the right breast or the left breast) is given to the acquired biopsydata.

The image-before-biopsy acquisition means 72 acquires, as an imagebefore biopsy, an image of a biopsy target part of a patient captured atthe date and time within a predetermined range before the biopsyexamination date and time. For example, since a mammography examinationis recommended to be performed every year, a plurality of mammographicimages of one patient are stored in the image database 5. The biopsy isperformed in a case in which the image analysis process 73 is performedfor a captured mammographic image and a shadow (hereinafter, referred toas a lesion candidate) determined to be abnormal is detected. Themammographic image used as a criterion for determining whether toperform a biopsy is the latest image before the biopsy date and time.However, it is considered that an image having a large time differencefrom the biopsy date and time, for example, an image captured one yearor more ago is not the image used as the criterion for determining thebiopsy. Therefore, for example, the image-before-biopsy acquisitionmeans 72 searches for the image of the biopsy target part capturedwithin a predetermined range (for example, within two months) before thebiopsy date and time among the images a biopsy target part (in thisembodiment, the breast) matched with the patient ID of the biopsy datafrom the image database 5. The searched image is considered to be themammographic image used as the criterion for determining whether toperform a biopsy immediately before an examination.

The image analysis process 73 is provided with a discriminator 79,discriminates a shadow that appears in the mammographic image using thediscriminator 79, detects lesion candidate regions, such as a tumor andcalcification, and acquires positional information related to thepositions of the lesion candidate regions. The positional informationmay be a coordinate value in the image, information in which ananatomical position is known, or information of an upper part, a middlepart, or a lower part of each organ. For example, in the case of thebreast, the positional information may be information of an inner upperpart, an inner lower part, an outer upper part, and an outer lower partof each of the left and right breasts. Hereinafter, the discriminator 79that detects the lesion candidate region is referred to as a firstdiscriminator.

Alternatively, the image analysis process 73 may detect whether themammographic image is an image in which a lesion is present with thediscriminator 79. Hereinafter, the discriminator 79 that detects whetherthe mammographic image is an image in which a lesion is present isreferred to as a second discriminator. For example, in a case in whichthe image is a mammographic image, a discriminator that can detectwhether a lesion is present in each of the CC images of the left andright breasts and the MLO images of the left and right breasts isprepared.

The scout image acquisition means 74 searches for a scout image havingan examination date matched with the patient ID of the biopsy data fromthe image database 5. The mammography apparatus 2 acquires, as the scoutimages, the partial images of a portion of the breast captured indifferent directions in a region in which abnormality is recognized in atarget part immediately before a biopsy. That is, the scout images aretwo partial images of a portion of the breast captured on the same dateas the examination date.

In a case in which a plurality of lesion candidate regions are detectedfrom the mammographic image by the image analysis process 73, thecorrespondence position determination means 75 searches for a lesioncandidate region having a shadow matched with the shadow appearing inthe scout images from the plurality of lesion candidate regions anddetermines a lesion candidate region at a correspondence positioncorresponding to the shadow included in the scout images. For the lesioncandidate region of the mammographic image matched with the shadowappearing in the scout images, the correspondence position determinationmeans 75 can determine a correspondence position corresponding to theposition where a biopsy has been performed from a plurality of lesioncandidate regions by finding a lesion candidate region of themammographic image which has a feature amount most matched with afeature amount extracted from the scout image. An example of the featureamount is a feature amount used to search for a similar image, such as ahistogram, scale-invariant feature transform (SIFT), or speeded-uprobust features (SURF).

In a case in which the biopsy result indicated by the biopsy dataindicates a lesion, the registration means 76 registers, as correctanswer data, the lesion candidate region obtained by performing theimage analysis process for the image before biopsy. In a case in whichone lesion candidate region is obtained by performing the image analysisprocess for the mammographic image and the biopsy result indicates alesion, the registration means 76 registers the lesion candidate regionas the correct answer data. In contrast, in a case in which a pluralityof lesion candidate regions are present, the correspondence positiondetermination means 75 specifies a correspondence position correspondingto the position where a biopsy has been performed in the mammographicimage from two scout images and registers, as the correct answer data, alesion candidate region matched with the correspondence position amongthe plurality of lesion candidate regions.

Alternatively, in a case in which the biopsy result indicates a lesion,the registration means 76 may register the image before biopsy as thecorrect answer data. For example, in a case in which the image beforebiopsy is a mammographic image, the registration means 76 registers, asthe correct answer data, the CC images of the left and right breasts andthe MLO images of the left and right breasts in the mammographic image.

The learning data storage unit 77 is provided in an auxiliary storagedevice such as a hard disk or a solid state drive (SSD). Alternatively,the learning data storage unit 77 may be provided in a NAS connected tothe network.

In addition, a correct answer data label is attached to a lesioncandidate region image and the lesion candidate region image is storedas learning data in the learning data storage unit 77. Alternatively, ina case in which the discriminator 79 determines an image indicating thata lesion is present, the correct answer data label is attached to theimage before biopsy acquired by the image-before-biopsy acquisitionmeans 72 and the image before biopsy is stored as the learning data.

The learning means 78 directs the discriminator 79 to relearn thecorrect answer data stored in the learning data storage unit 77. Theimage analysis process 73 detects a lesion candidate region using thediscriminator 79 that has learned the correct answer data in advance.The image analysis process 73 directs the first discriminator 79 torelearn the correct answer data which is the result of the determinationof whether the detected lesion candidate region is a lesion from thebiopsy result or directs the second discriminator 79 to relearn thecorrect answer data which is the result of the determination of whetherthe image before biopsy is an image indicating that a lesion is presentfrom the biopsy result. This configuration makes it possible to furtherimprove the discrimination performance. It is preferable to direct thediscriminator 79 to learn the correct answer data and to update theimage analysis process 73 in a case in which a certain amount oflearning data is accumulated.

Next, a method in which the mammography apparatus captures mammographicimages and performs a biopsy and the image processing server createslearning data on the basis of the biopsy result will be described. FIG.7 is a flowchart illustrating an example of the flow of an imagingprocess of the mammography apparatus. FIG. 8 is a flowchart illustratingan example of the flow of a learning data creation process of the imageprocessing server (learning data creation support apparatus). The flowof a process from the capture of a mammographic image to the creation oflearning data will be described according to the flowcharts illustratedin FIGS. 7 and 8.

First, the flow of the capture of an image will be described accordingto the flowchart illustrated in FIG. 7. The breast M of a patient isplaced on the imaging table 14 of the mammography apparatus 2 in orderto capture a mammographic image. The operator moves the arm 13 in thevertical direction with the operation unit 28 to adjust the height ofthe arm 13 according to the height of the breast M of the patient. Inaddition, the operator operates the operation unit 28 to direct the armcontrol unit 31 to incline the arm 13 according to whether to capturethe images of the breast M in a mediolateral oblique (MLO) direction ora craniocaudal (CC) direction. In a case in which the operator positionsthe imaging table 14 at a height and an inclination angle suitable forimaging, the compression plate controller 34 compresses the breast untilpressure against the compression plate 18 reaches a predetermined value.In addition, after the operator inputs various imaging conditions to themammography apparatus 2, a command to start imaging is input. Theradiation emitting unit 17 emits radiation according to the imagingconditions and the flat panel detector 15 acquires a mammographic image(Step ST1). Accessory information based on the DICOM standard is givento the acquired mammographic image and the mammographic image istransmitted to the image database 5. In addition, the mammographic imageis transmitted to the image processing server 7 (Step ST2) and the imageprocessing server 7 performs the image analysis process 73. Themammography apparatus 2 temporarily ends the capture of the images ofthe breast M and the determination result in Step ST3 is “NO” until alesion candidate which is a biopsy target is received from the imageprocessing server.

In the image processing server 7, the image analysis process 73 isperformed for the mammographic image to detect lesion candidate regionssuch as a tumor and calcification. The detected lesion candidate regionsand the positional information thereof are transmitted as the analysisresults from the image processing server 7 to the radiologist WS 3. Inthe radiologist WS 3, marks are put on the lesion candidate regions ofthe mammographic image on the basis of the received analysis results andthe lesion candidate regions are displayed. The radiologist interpretsthe mammographic image. In a case in which the radiologist designates alesion candidate region requiring a biopsy among a plurality of lesioncandidate regions displayed on the display with, for example, a mouse,the positional information of the designated lesion candidate region istransmitted from the radiologist WS 3 to the image processing server 7.For example, in a case in which a lesion candidate region is present inthe inner upper part of the right breast, positional informationindicating the inner upper part of the right breast is transmitted fromthe radiologist WS 3 to the image processing server 7. In addition, theimage processing server 7 transmits the positional information of thelesion candidate region which is a biopsy target T to the mammographyapparatus 2.

Then, in a case in which the mammography apparatus 2 receives thepositional information of the lesion candidate region from the imageprocessing server 7, the determination result in Step ST3 is “YES”. In acase in which the breast M of the same patient is placed on the imagingtable 14 in the same procedure as described above in order to perform abiopsy using the mammography apparatus 2, the compression platecontroller 34 compresses the breast of the patient with the compressionplate 18. Before needling, for example, two scout images obtained bycapturing the images of the inner upper part of the right breast in twodirections so as to include the lesion candidate region of the breast Mwhich is the biopsy target T on the basis of the positional informationof the received lesion candidate region as illustrated in FIG. 5 areacquired (Step ST4). The mammography apparatus 2 acquires thethree-dimensional positional information of the target T from the twoscout images (Step ST5). The needle position controller 35 of the biopsyunit 26 moves the tip of the biopsy needle 21 to the position of thetarget T on the basis of the three-dimensional positional informationand the biopsy needle unit 22 inserts the biopsy needle 21 into thebreast (Step ST6).

The case in which the positional information for capturing the scoutimages is received from the image processing server 7 has been describedabove. However, the operator of the mammography apparatus 2 may check,for example, an interpretation report and manually input an imagingposition to the mammography apparatus 2 such that the scout images arecaptured.

In addition, for example, a pathologist performs a pathology examinationfor the tissues obtained by biopsy and the pathology examination WS 8makes a pathology examination report including biopsy data. Thepathology examination report that has been made, a patient ID, apathologist ID, a lesion name, the positional information of a lesion,and the doctor's opinion are registered in the pathology report database9.

Next, the flow of a learning data creation process of the imageprocessing server 7, that is, the learning data creation supportapparatus 7 will be described according to a flowchart illustrated inFIG. 8. In this embodiment, a case in which, if the pathologyexamination WS 8 makes a pathology examination report, the patient ID istransmitted to the learning data creation support apparatus 7, amammographic image and biopsy data are acquired using the patient ID,and learning data is created will be described. In this embodiment, acase in which the user selects whether to register the image of thelesion candidate region as the learning data or to register the imagebefore biopsy as the learning data in advance will be described. First,a case in which the image of the lesion candidate region is set to beregistered as the learning data will be described.

First, in a case in which the patient ID on which a pathologyexamination report has been made is received from the pathologyexamination WS 8, the determination result in Step ST11 is “YES”. Thebiopsy data acquisition means 71 searches for the pathology examinationreport corresponding to the patient ID from the pathology reportdatabase 9 and acquires biopsy data from the pathology examinationreport (Step ST12). In a case in which the biopsy result indicated bythe biopsy data is a lesion, the determination result in Step ST13 is“YES”. The image-before-biopsy acquisition means 72 acquires themammographic image obtained by capturing the image of the breast whichis a biopsy target part of the patient ID from the image database 5(Step ST14). It is highly possible that the same patient was subjectedto the mammography examination a plurality of times in the past and aplurality of mammographic images were stored in the image database 5.However, since whether to perform a biopsy for the breast of the patientis determined on the basis of the mammographic image capturedimmediately before the biopsy, an image captured at the date and timewhich is before the examination date and time and is closest to theexamination date and time is acquired from the mammographic imagescorresponding to the patient ID of the patient who has been subjected toa biopsy (Step ST15). However, the biopsy is not determined on the basisof, for example, the old images captured one year or more ago.Therefore, in a case in which the acquired mammographic image is within,for example, two months before the biopsy date and time, thedetermination result in Step ST16 is “YES”.

On the other hand, in a case in which the acquired mammographic image isnot within, for example, two months before the biopsy date and time, thedetermination result in Step ST16 is “NO”. In this case, since it isdifficult to find a mammographic image for the biopsy result,registration is not performed. Until the determination of whether thereis correct answer data for all of the biopsy data ends, thedetermination result in Step ST22 is “NO” and the process returns toStep ST11. The learning data creation support apparatus 7 waits for theinput of the next patient ID.

In a case in which the determination result in Step ST16 is “YES”, it isdetermined whether to register the lesion candidate region or toregister the image indicating that a lesion is present (Step ST17). In acase in which the image of the lesion candidate region is registered asthe learning data, the determination result in Step ST18 is “YES” andthe lesion candidate region obtained by performing the image analysisprocess 73 for the mammographic image closest to the examination dateand time is acquired (Step ST19). In addition, the scout imageacquisition means 74 acquires the scout images of the same patient IDcaptured on the biopsy date from the image database 5 (Step ST20). In acase in which a plurality of lesion candidate regions are detected fromthe mammographic image by the image analysis process 73, thecorrespondence position determination means 75 determines whether thereis a shadow of a lesion candidate region matched with the shadowappearing in the scout image (Step ST21).

In a case in which a lesion candidate region matched with the scoutimage is found, the determination result in Step ST22 is “YES” and theregistration means 76 registers the lesion candidate region as thecorrect answer data in the learning data storage unit 77 (Step ST23). Onthe other hand, in a case in which a lesion candidate region matchedwith the scout image is not found, the determination result in Step ST22is “NO” and the correct answer data is not registered in the learningdata storage unit 77. In addition, until the determination of whetherthere is correct answer data for all of the biopsy data ends, thedetermination result in Step ST24 is “NO” and the process returns toStep ST11. The learning data creation support apparatus 7 waits for theinput of the next patient ID.

Then, in a case in which another patient ID is input, the determinationresult in Step ST11 is “YES” and the biopsy data acquisition means 71searches for a pathology examination report corresponding to the patientID from the pathology report database 9 and acquires biopsy data fromthe pathology examination report (Step ST12). However, in a case inwhich the biopsy result indicated by the biopsy data is not a lesion(the determination results in Step ST13 and Step ST24 are “NO”), theprocess returns to Step ST11. The learning data creation supportapparatus 7 waits for the input of the next patient ID.

As described in detail above, in a case in which the biopsy resultindicates a lesion, a shadow corresponding to the biopsy result is morelikely to be a lesion, as compared to a case in which the biopsy resultdoes not indicate a lesion. Therefore, the learning data creationsupport apparatus 7 searches for the image captured before a biopsy andregisters a lesion candidate region detected from the searched image asthe correct answer data. As a result, it is possible to generatelearning data without a complicated operation.

The case in which a plurality of lesion candidate regions are detectedfrom the mammographic image has been described above. However, in a casein which only one lesion candidate region has been detected, thelearning data creation support apparatus 7 may not determine whether ashadow in the lesion candidate region is matched with the shadowappearing in the scout image. In a case in which the biopsy resultindicates a lesion, the detected lesion candidate region may beregistered as the correct answer data in the learning data storage unit77.

Next, a case in which the image before biopsy is set to be registered asthe learning data will be described. A process from Step ST11 to StepST16 is performed in the same way as described above to find amammographic image which is an image before biopsy for the biopsyresult. Then, it is determined whether to register the lesion candidateregion or to register the image indicating that a lesion is present(Step ST17). In a case in which the image indicating that a lesion ispresent is registered as the learning data, the determination result inStep ST18 is “NO” and the mammographic image which is the image beforebiopsy for the biopsy result is registered as the correct answer data inStep ST25. Then, the process proceeds to Step ST24. The process fromStep ST11 to Step ST24 is repeated until the determination of whetherthere is correct answer data for all of the biopsy data ends.

Next, the flow of a process for causing the discriminator 79 of theimage analysis process to learn the correct answer data accumulated inthe learning data storage unit 77 will be described with reference tothe flowchart of a learning process illustrated in FIG. 9.

In a case in which a certain amount of learning data is accumulated, thelearning means 78 directs the discriminator 79 to perform relearning.First, it is determined whether to learn the lesion candidate region orto learn the image indicating that a lesion is present (Step ST30). In acase in which the lesion candidate region is learned, the determinationresult in Step ST31 is “YES” and the learning means 78 extracts thecorrect answer data of the image of the lesion candidate region from thelearning data storage unit 77 (Step ST32) and directs the firstdiscriminator 79 to perform relearning (Step ST33). The discriminator 79that has performed relearning is incorporated into the image analysisprocess and the image analysis process is reinstalled in the imageprocessing server 7.

On the other hand, in a case in which the image indicating that a lesionis present is learned, the determination result in Step ST31 is “NO” andthe learning means 78 extracts the correct answer data of themammographic image which is the image before biopsy from the learningdata storage unit 77 (Step ST40) and directs the second discriminator 79to perform relearning (Step ST41). The discriminator 79 that hasperformed relearning is incorporated into the image analysis process andthe image analysis process is reinstalled in the image processing server7.

As such, since the correct answer data created on the basis of thebiopsy result is learned, it is possible to improve the discriminationperformance of the image analysis process.

As described in detail above, since the discriminator performsrelearning, it is possible to improve the discrimination performance ofthe image analysis process and to repeat the creation of learning datausing the result of the image analysis process and the biopsy result.Since the learning data is created in this way, it is possible tocontinuously improve the performance of the image analysis process.

The case in which the correct answer data is created has been describedabove. However, the region in which the biopsy result is a lesion andthe result of the image analysis process is determined to be a normalshadow may be registered as the incorrect answer data in the learningdata storage unit 77 and the discriminator may learn both the correctanswer data and the incorrect answer data.

The breast has been described above as an example. The invention may beapplied to other organs (for example, the stomach, the intestines, theliver, the kidney, or the lungs) and the correct answer data or theincorrect answer data may be created according to the biopsy results ofeach organ.

In the above-described embodiment, the hardware structure of aprocessing unit that performs various processes of the learning datacreation support apparatus is the following various processors. Thevarious processors include a CPU which is a general-purpose processorthat executes software (program) to function as various processingunits, a programmable logic device (PLD) which is a processor whosecircuit configuration can be changed after manufacture, such as afield-programmable gate array (FPGA), and a dedicated electric circuit,such as an application specific integrated circuit (ASIC), which is aprocessor having a dedicated circuit configuration designed to perform aspecific process.

One processing unit may be formed by one of the various processors ormay be formed by a combination of two or more processors of the sametype or different types (for example, a combination of a plurality ofFPGAs or a combination of a CPU and an FPGA). In addition, a pluralityof processing units may be formed by one processor. A first example ofthe configuration in which the plurality of processing units are formedby one processor is an aspect in which one or more CPUs and software arecombined to form one processor and the processor functions as aplurality of processing units. A representative example of the aspect isa computer such as a client apparatus or a server. A second example ofthe configuration is an aspect in which a processor that implements allof the functions of a system including the plurality of processing unitswith one integrated circuit (IC) chip is used. A representative exampleof the aspect is a system-on-chip (SoC). As such, the hardware structureof various processing units is formed by using one or more of thevarious processors.

EXPLANATION OF REFERENCES

1: medical information system

2: modality (mammography apparatus)

3: radiologist workstation

4: diagnosis and treatment department workstation

5: image database

6: interpretation report database

7: image processing server (learning data creation support apparatus)

8: pathology examination workstation

9: pathology report database

10: network

11: base

12: axis

13: arm

14: imaging table

15: flat panel detector

16: radiation accommodation unit

17: radiation emitting unit

18: compression plate

19: moving mechanism

20: support unit

21: biopsy needle

22: biopsy needle unit

24: moving mechanism

25: opening portion

26: biopsy unit

28: operation unit

31: arm control unit

34: compression plate controller

35: needle position controller

71: biopsy data acquisition means

72: image-before-biopsy acquisition means

73: image analysis process

74: scout image acquisition means

75: correspondence position determination means

76: registration means

77: learning data storage unit

78: learning means

79: discriminator

M: breast

P1, P2: irradiation position

T: target

Img1, Img2: scout image

z: distance

What is claimed is:
 1. A learning data creation support apparatuscomprising: a biopsy data acquisition unit that acquires biopsy dataindicating a result of a biopsy performed for a patient; animage-before-biopsy acquisition unit that acquires, as an image beforebiopsy, an image of a biopsy target part of the patient captured at adate and time within a predetermined range before an examination dateand time of the biopsy; and a registration unit that, in a case in whichthe biopsy result indicated by the biopsy data indicates a lesion,registers, as correct answer data, an image of a lesion candidate regionobtained by performing an image analysis process for the image beforebiopsy or the image before biopsy.
 2. The learning data creation supportapparatus according to claim 1, wherein, in a case in which the image ofthe lesion candidate region is registered, the registration unitspecifies a correspondence position corresponding to a position wherethe biopsy has been performed in the image before biopsy from twopartial images of the target part captured in different directions, andin a case in which the biopsy result indicates a lesion, theregistration unit registers, as the correct answer data, an image of alesion candidate region at a position matched with the correspondenceposition among positions of a plurality of lesion candidate regionsobtained from the image before biopsy.
 3. The learning data creationsupport apparatus according to claim 1, further comprising: a learningunit that, in a case in which the image of the lesion candidate regionis registered, detects a lesion candidate region, using a discriminatorthat has learned the correct answer data of the image of the lesioncandidate region, and directing the discriminator to relearn the correctanswer data of the registered image of the lesion candidate region. 4.The learning data creation support apparatus according to claim 2,further comprising: a learning unit that, in a case in which the imageof the lesion candidate region is registered, detects a lesion candidateregion, using a discriminator that has learned the correct answer dataof the image of the lesion candidate region, and directing thediscriminator to relearn the correct answer data of the registered imageof the lesion candidate region.
 5. The learning data creation supportapparatus according to claim 1, further comprising: learning unit that,in a case in which the image before biopsy is registered, detectingwhether an image indicating a lesion is present, using a discriminatorthat has learned the correct answer data of the image before biopsy, anddirecting the discriminator to relearn the correct answer data of theregistered image before biopsy.
 6. The learning data creation supportapparatus according to claim 2, further comprising: learning unit that,in a case in which the image before biopsy is registered, detectingwhether an image indicating a lesion is present, using a discriminatorthat has learned the correct answer data of the image before biopsy, anddirecting the discriminator to relearn the correct answer data of theregistered image before biopsy.
 7. The learning data creation supportapparatus according to claim 1, wherein the image of the biopsy targetpart is a mammographic image.
 8. The learning data creation supportapparatus according to claim 2, wherein the image of the biopsy targetpart is a mammographic image.
 9. The learning data creation supportapparatus according to claim 3, wherein the image of the biopsy targetpart is a mammographic image.
 10. The learning data creation supportapparatus according to claim 4, wherein the image of the biopsy targetpart is a mammographic image.
 11. A learning data creation supportmethod performed in a learning data creation support apparatus, themethod comprising: a biopsy data acquisition step of acquiring biopsydata indicating a result of a biopsy performed for a patient; animage-before-biopsy acquisition step of acquiring, as an image beforebiopsy, an image of a biopsy target part of the patient captured at adate and time within a predetermined range before an examination dateand time of the biopsy; and a registration step of, in a case in whichthe biopsy result indicated by the biopsy data indicates a lesion,registering, as correct answer data, an image of a lesion candidateregion obtained by performing an image analysis process for the imagebefore biopsy or the image before biopsy.
 12. A non-transitory computerrecording medium storing a learning data creation support program thatcauses a computer to perform: a biopsy data acquisition step ofacquiring biopsy data indicating a result of a biopsy performed for apatient; an image-before-biopsy acquisition step of acquiring, as animage before biopsy, an image of a biopsy target part of the patientcaptured at a date and time within a predetermined range before anexamination date and time of the biopsy; and a registration step of, ina case in which the biopsy result indicated by the biopsy data indicatesa lesion, registering, as correct answer data, an image of a lesioncandidate region obtained by performing an image analysis process forthe image before biopsy or the image before biopsy.
 13. A learning datacreation support apparatus comprising: a memory that stores commands forcausing a computer to perform processes; and a processor that isconfigured to execute the stored commands, wherein the processoracquires biopsy data indicating a result of a biopsy performed for apatient, acquires, as an image before biopsy, an image of a biopsytarget part of the patient captured at a date and time within apredetermined range before an examination date and time of the biopsy,and registers, as correct answer data, a lesion candidate regionobtained by performing an image analysis process for the image beforebiopsy in a case in which the biopsy result indicated by the biopsy dataindicates a lesion.